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A prototype computer-assisted teaching training 
system (CATTS) vas developed to help train special education 
personnel, CATTS is a comprehensive system which produces systematic 
observation, real-time analysis, storage, and feedback of specific 
observation coding data relevant to special education classroom 
teacher-pupil interaction^. An investigation demonstrated the 
relative effectiveness of CATTS instantaneous (visual) and delayed 
(printout) feedback in increasing various cognitive and management 
behaviors of preservice teacher-trainees in a controlled classroom 
setting. It was found that CATTS is a versatile and comprehensive 
delivery system which can be applied in many ways to analyze 
real-time process variables within behavior training fields. Two 
statistical tables and three graphs include a schematic diagram of 
the arrangement of a CATTS station, (Author/JY) 
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The Effectiveness of a Computer-Assisted Teacher Training System 
(CATTS) in Generating Specific Teacher Behaviors in a 
Preservice College Teaching Environment 
Merrill C. Sitko, Molvyn I. SenCTel, and Jerry Olson 
Center for Innovation in Teaching the Handicapped 
Indiana University 
Abstract 

The present paper discusses the application of a computerized feedback 
system to teacher training and the study of teacher-pupil interactive be- 
havior in classrooms. A prototype computer assisted teaching training system 
(CATTS) is described in detail. Moreover, the efficacy of CATTS as a train- 
ing system is empirically demonstrated. Tlie paper specifically discusses an 
investigation which demonstrated the relative effectiveness of CATTS instan- 
taneous (visual) and delayed (printout) feedback in increasing various cogni- 
tive and management behaviors of preservice teacher-trainees in a controlled 
laboratory classroom setting. The feasibility of utilizing the prototype 
computerized feedback system for in situ training in other training environ- . 
ments involving behavior analysis and behavior modification is also indicated. 
In essence, CATTS is a versatile and comprehensive delivery system which 
cwi'be applied in many ways to analyze real-time process variables within 
behavior training fields. 
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Our current plan for presentation of this paper is to develop our 
talk around a slide, shovr format. The presentation will be roughly broken 
into three sections. Tlie initial section will be a description of the 
Computer Assisted Teacher Training System (CATTS) supplemented with slides 
showing' the general configuration of the laboratory, coding and teaching 
stations. The second section will describe the teacher training program 
and the third section, again supplemented by slides and sample printouts, 
x^ill describe the analysis and results. 

The presentation will not be a verbatim reading of the paper, but 
instead will be a discussion of the application of the CATTS delivery 
system to our teacher training focus and its potential application to 
other behavioral teaching programs. 
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Educators have, in the past decade, participated in the technological 
revolutions provoked by advances in computer development. Most large school 
systems use computers for scheduling, general accounting, grading and 
other automatic functions which previously demanded the long and arduous 
labor of relatively skilled personnel. In addition, the advent of "'real- 
time'' systems and shared time arrangeinents has brought the capabilities of 
rapid analysis and feedback directly into the learning situation through 
programmed instructional techniques and various audio-visual approaches. 
Computer-monitored instructional programs are currently attempting to reach 
large numbers of pupils with fewer numbers of teachers. Others seek to 
individualize instruction for specific children. Undoubtedly, as such 
efforts progress and cost factors are controlled, we will be faced with 
the reality of an educational technology revolution in education within 
the coming decade. 

The question whicli arises then is "Can we presently utilize computer 
technology for preparing teachers to work effectively with children in 
both special and/ or regular class settings?" To our knowledge, little 
attempt has been made to explore and evaluate the potential contribution 
of computers in clarifying training objectives and improving teacher com- 
petencies in existing teacher education programs. Hence, there is a 
current need for the exploitation d. computer technology in teacher train- 
ing programs in education. 

Research and developmental activities directed toward realizing a 
cost-effective Computer-Assisted Teacher Training System (CATTS) for train- 
ing special education personnel have continued at the Center for Innovation 
in Teaching the Handicapped (CITH), Indiana University, for the pact four 
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years under the direction of M. I, Semniel (1972, a, b), CATTS is con- 
ceptualized as a closed-loop cybernetic system capable of producing con- 
tinuous instantaneous and/or delayed feedback of relevant teacher-pupil inter- 
action data to a teacher trainee in the classroom, so that modification of 
behavior can be realized through regulatory teaching moves in accordance 
with predetermined training objectives. Tlie comprehensive system is de- 
signed to produce a feasible, cost-effective means of systematic observation, 
real-time analysis^ storage, and feedback of specific observation-coding data 
relevant to special education classroom teadier-pupil interactions. An 
immediate visual feedback delivery system and corresponding data summaries 
have been developed for iin situ and after-session feedback of relevant 
teacher-pupil variables in practicum teaching environments. 

CATTS SYSTEM CONFIGURATION 

Tlie system configuration of CATTS is visualized as consisting of three 
interdependent stations: Teaching Station, Observation -Coding Station, 
and Analysis-Encoding Station. Figure 1 illustrates this configuration 
with a schematic diagram of the present CATTS installation at CITH. 

Teaching station . Hie CATTS Teaching Station consists of classroom(s) 
which accommodate various auditory and visual feedback devices. Space is 
also available for visual observation and an input device to direct com- 
puter entry of classroom behavior. Visual feedback devices are placed so 
that the teacher can have access to the feedback info!rmation during teach- 
ing with a minimum of classroom disruption. T]\ese various feedback dis- 
play devices are controlled directly by the computer. 

Application of visual displays into the Teaching Station vary from the 
closed-circuit televised images currently employed, which originate from a 



cathode-ray tube (CRT) computer display to various external visual devices, 
which convey feedback information to the classroom by changing light patterns. 

Observation - Coding Station. TTie Observation -Coding Station provides 
the link between the events occurring in the classroom and the computer 
analysis of these events. Visual observation of classrooms takes place 
within the classroom itself, within an observation room adjoining the class- 
room, or through a closed-circuit television connection. Three methods 
of data input are employed by the CATTS system: a directly-connected 
coding box device; a Toucli Tone telephone access system; and a portable 
recording unit for off-line data collection. The first two methods permit 
real-time data collection for instantaneous feedback applications. Tlie 
third method extends the CATTS system's data collection ability beyond 
the physical limitations of any direct computer connection into the field, 
which allows baseline and delayed feedback observations. 

At present, the computer-connected coding device with which the observf«r 
enters his observations consists of a small hand-held box with 12 Touch Tone 
configured pushbuttons. The other direct input connection is the Touch Tone 
telephone system. Currently, the CATTS system can accept data independently 
and simultaneously from 12 directly connected coding devices. This input 
time-sharing capability allows simultaneous data collection from various 
locations to fulfill classroom observation requirements for independent 
projects. 

A nalysis Encoding Station . Tlie Analysis-Encoding Station contains a 
PD1>-12 digital computer and associated computing hardware required for the 
on-line processing of pushbutton-coded data which are gathered and trans- 
mitted from Observation Coding Stations. In addition to processing the 



incoming data, the computer system is configured to allow for the control 
of various display devices used in Teaching Stations and provides for hard- 
copy printouts, storage, and transfer of data to remote computer systems 
for further analysis. It is at v ds station that the operator or project 
director initiates the computer program options available in CATTS. The 
teleprinter console, through software program control, permits the operator 
to select specific CATTS programs or options that satisfy various project 
data collection and feedback requirements. 

The selection of the mode and content of feedback to the teacher at the 
Teaching Station is initiated from the console. If a CRT display is chosen 
as the method for feedback, the operator determines the content of the dis- 
play by assigning the incoming data, by code, to different computational 
functions for the computer to calculate and display as feedback. The nature 
of the display also is selected from the console and allows feedback infoima- 
tion to be presented either in alpha-numeric or graphic form. 

Upon completion of an observation session, the- raw data are stored 
upon local computer magnetic tape units and become accessible for various 
data analysis applications. Analysis programs are available on both the 
PDP-12 computer and Indiana University's remote computer facility. Prior 
to remote computing center data analysis applications, the data are trans- 
ferred by a telephone connection into permanent disk and tape storage files 
whicli are accessible by either batch or time-sharing analysis programs. 

The associated PDP-12 teleprinter, which also serves as the communica- 
tions link to the CATTS program, provides hard-copy data summary printouts 
for instant inspection throughout the data gathering stage. These printouts 
provide such information as the frequency and event times of coded tallies 
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and the actual feedback functions currently being displayed to the teacher • 
At the end of a real-time session, a selection of various data suanary 
programs and printouts is available for delayed feedback applications. 

PROJECT DESCRIPTION 

In order to empirically demonstrate the efficacy of the CATTS system 
the present project was undertaken. This project attempted to demonstrate 
the relative effectiveness of computerized instantaneous and delayed feed- 
back in generating specific teacher behaviors in a preiervice teaching en- 
vironment. The project built on previous developmental work involving the 
CATTS system. Specifically thef iJUrpose of the project was to demonstrate 
the effect of CATTS immediate visual and delayed (printout) feedback on 
increasing various cognitive and management behaviors of teacher-trainees 
in a controlled laboratory classroom setting. In all the study entailed 
three major phases; first, participation by trainees in observation system 
training; second baseline of trainees' cognitive demands and behavioral 
control strategies in the classroom; third, measurement of trainees* cogni- 
tive demands and behavioral control strategies under CATTS instantaneous 
and/or delayed feedback conditions. 

In the first or Discrimination Phase of the project the 27 special 
education preservice trainees learned two category observation systems en- 
titled COG-STRAT and MAN-STRAT. Categories on the COG-STRAT included teacher 
and pupil cognitive styles of interaction, while MAN-STRAT categories focused 
on pupil on- and off-task behavior and strategies of teaching management of 
pupil behavior. All tr&inees were trained to code all categories on the two 
systems. Acquisition of observation-coding skills was facilitated by the 
computer-aided training device called DITRMA. The total time for trainee 
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acquisition of the COG-STRAT system varied from 8 to 10 hours, while MAN-STRAT 
training took slightly longer. Following observer training, both criterion- 
related and intracoder measures were obtained respectively for COG-STRAT and 
MAN-STRAT observers via videotaped criterion tests. All trainees achieved 
initial criterion-related agreement (Inter-rater reliability) scores ^.85, 
In addition to initial criterion-related agreement, initial intraobserver 
agreement measures were obtained for both systems. All trainees exceeded 
the preset standard of .75 for both pupil and category identification. In 
fact almost all exceeded a .85 standard. Maintenance checks for criterion- 
related agreement were similarly performed about one-third the way through 
the study, except different videotapes were used. The majority of the trainees 
exceeded the .85 standard on the maintenance check. 

In each of the two teaching phases (i*e.. Baseline and Feedback), 
trainees taught language art lessons they had prepared. Lesson objectives 
that trainees used for lesson plan preparation were stated in behavioral 
terms. Expected pupil terminal behaviors indicating the pupils attainment 
of a concept or ability to perform a task were the basis for a given objec- 
tive. The trainees were required to submit a written lesson plan for each 
30-minute lesson they taught one week prior to teaching for comment and evalu- 
ation. In the teaching phase of the study, each practicum student was randomly 
assigned to teach the same group of EMR children, once each week for the dura- 
tion of the study. There were four to five children in each group and there 
were three such groups. These groups were drawn from a class of EMR children 
whose regular classroom was located in a classroom adjacent to the Teacher 
Education Laboratory at CITH. The grouping of pupils for instruction by 
trainees was based upon the similarity of instructional needs and the social 



ERIC 



3 



7 

maturity of the pupil. Each group of pupils received a total of one hour of 
supplementary reading instruction by the teacher trainees per day. The same 
group of children was taught by one trainee for half an hour and by another 
trainee for half an hour. In addition each trainee coded another trainee 
in his group at least once a week on the COG-STRAT system. Coders were 
randomly assigned to teachers such that each teacher was observed by a dif- 
ferent coder in his group each time. Assignmant of trainees to day of teach- 
ing was random. Teaching room assignment was also rotated between three 
classrooms in the TEL facility. Trainees coded classroom interaction on 
button boxes which were hooked up to a PDP-12 computer located in the TEL, 
Coding for each lesson continued for 30 minutes after which the computer auto- 
matically shut off the coding terminals. 

A single organism multiple baseline across subjects design was selected 
for the two teaching phases of the project. In the baseline condition, trainees 
taught the lessons they had prepared without receiving feedback of any kind. 
The number of lessons taught without feedback varied from trainee to trainee • 
The baseline teaching always included at least the first three lessons and 
varied up to the first six lessons taught. Assignment of trainees to variable 
baseline teaching trials was random. The second controlled condition was 
teaching after the trainee received computer printout information or feedback 
on his baseline teaching performance and a set of instructions in modular 
form on how to read the computer printout. This feedback was based on both 
COG-STRAT and MAN-STRAT categories. 

After their baseline trials were completed, the trainees were provided 
with printouts which summarized their performance across frequency, time and 
rate dimensions for each of the COG-STRAT and MAN-STRAT observation categories. 
They were then instructed in evaluation procedures for using the printout 
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information to graph the percent frequency of occurrence of behaviors for 
each of the observation ays tea categories. They were also assigned the 
task of graphing all categories in the baseline period. At this stage the 
trainees were also required to evaluate their baseline teaching on the basis 
of their graphs. They then selected and listed in order of personal preference 
three of the observation system categories from COG-STRAT and MAN-STRAT systems 
that they judged as most important to their teaching and wished to work on for 
improving their teaching performance. Within a week of receiving baseline 
information, the trainees were required to select a single behavioral category 
for subsequent improvement. The final selection was arrived at in consulta- 
tion with a staff member. This choice became, in most cases, the category 
that was targeted for improvement in subsequent lessors. The trainee also 
had to indicate whether he wanted to accelerate or deccelerate the percent 
frequency of occurrence of the target behavioral category in subsequent lessons. 
In most cases the trainees first choice was an acceptable behavioral category 
indicative of a relatively stable baseline performance on that category during 
baseline. All of these activities were carried out after the baseline period 
and before the trainees taught with feedback. 

Following these procedures, the trainees were randomly assigned to one 
of two feedback conditions. Half of the trainees were assigned to a printout 
only feedback group, and half to a scope and printout feedback group. The 
scope feedback group was presented with a self instructional module and in- 
structions from a staff member on how to interpret CATTS video scope feedback. 
This step was performed prior to the trainees teaching with feedback. During 
the feedback phase, the weekly lessons continued as during baseline and in 
addition all trainees received printouts on their latest lesson within two 
hours after teaching. The group receiving instantaneous video or scope 
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feedback always had a video monitor in their classroom which showed the 
cumulative percent frequency of occurrence of the targeted behavioral cate- 
gory that the trainee had chosen for teaching behavior change. Hence, this 
group obtained instantaneous or immediate information on the criterion teach- 
ing behavior in situ while teaching. In addition they also received printouts 
summarizing their les8o:i8 as did the other group. Both groups were required 
to graph the percent frequency for the targeted category after the printout 
was received. This dki'ca was recorded on a cumulative individual graph for 
the category and included the trainees baseline percentages. In addition, 
all trainees kept a running record of the percent frequency in category for 
each successive lesson by entering the information for the previous lesson 
on a graph in the laboratory. 

The instantaneous video feedback group was required to fill out a post 
teaching questionnaire after each lesson they taught. It was expected that 
the questionnaire would serve to reinforce the trainees attention to the 
scope during teaching. Among the items in the questionnaire was one in 
which the trr.inee was required to estimate the number of times he looked 
at the video monitor. Another question required the trainee to reconstruct 
the video feedback display for the entire 30-minute lesson. The number of 
lessons taught with feedback varied from trainee to trainee. The feedback 
teaching included at least four lessons and varied up to seven lessons 
taught depending on the number of lessons taught during the baseline phase. 
The teaching phases of the project covered a total period of ten weeks, 
ending May* 1973. 

RESULTS 

The design used to analyze the CATTS project data was a repeated measures 
ANOVA design with two between block factors; Groups (G) and i\eatments (T), 
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and one within block factor, Periods (P) . The original deaign contained a 
fourth factor, different variations of baseline and treatment trial conbina- 
tiona, but due to inconaiatent teaching schedules and name missing observa- 
tions, this baseline /treatment factor was never fully completed. Therefore, 
each subject's trials were collapsed within the various baseline and treat- 
ment periods and one average performance rate score per period was calculated. 

The rate of responding on the trainees' chosen feedback category was 
selected for the dependent measure employed in the analyses. This criterion 
measure was calculated by dividing the frequency of occurrence of the feed- 
back category during any one session by the time of that session. 

Table 1 contains the analysis of variai e source tables for the feedback 
category as well as the second and third choice categories. F ratios for the 
feedback category indicated a significant finding in the period (P) main 
effect (p <.01). An examination of the means in Table 2 and Figure 2 reveal 
that treatment trials were significantly higher than baseline trials by a 
2.8:1 ratio. In addition the Treatment by Period interaction (TP) was also 
significant (p <.05). 

Due to this significant interaction a simple main effects analysis 
(Kirk, 1968) was performed on T and P to qualify these main effects. The 
results of the simple effects analysis are also indicated iu Table 1. Figure 
1 Illustrates the TP interaction plot. 

As expected, the qualification of the Treatment effect (T) indicated 
that the scope and printout feedback condition (T2) and the printout only 
feedback condition (T^) were not significantly different during baseline 
trials (Pj^), but did reach significance (p <^ .05) during the treatment trials 
(P2) by an approximate 2:1 ratio. The Period effect (P) is qualified by 
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shoving that the printout only treatment group (Tj^) did not significantly 
increase their mean rate of responding between baseline (Pj^) and treatment 
(Pj) trials, while the CATTS scope display and printout treatment group 
(T^) increased significantly (p .10) between baseline and treatment trials 
by a 3.7:1 ratio. 

To further sipport the simple main effects interpretation, identical 
results were achieved by performing Tukey post hoc analyses (Winer, 1971) 
on the means which are plotted in Figure 1. 

As indicated above, prior to the trainees selection of their criterion 
feedback category, they initially chose three teaching behaviors that they 
specified they would like to work on in the classroom. From this list of 
three, the final feedback category was selected. As a contrast to the 
selected feedback category, identical analyses were performed on the second 
and third choice behaviors that were not included in the concentrated CATTS 
feedback program. These second and third choice behaviors were not dis- 
played on the scope instantaneous feedback display, but were included on 
the printout given to both treatment groups together with all of the other 
behavior categories coded in the classroom. 

An examination of Table 1 variance components for second and third 
choice categories revealed no significant main effect differences or inter- 
actions present in the data. Hence, only the criterion feedback category 
selected from the original chosen three categories was found to Improve 
significantly from baseline to treatment. Figure 3 (a and b) graphically 
illustrate the mean rate increase from baseline and performance differences 
from baseline between the three feedback choice categories. 

F max tests (Kirk, 1968) were performed on the error terms for all 
three categories found in Table 1. F max ratios indicated that both the 
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within lubjecti and between lubjects variances error tema for the feedback 
category were found to be made up from homogeneous within cell components » 
The within subjects error term for the second choice category was also found 
to be homogeneous, The F max ratios for the remaining error terms rejected 
the homogeneity assumption. The application of this test qualifies the 
interpretation of the second and third choice categories but lends support 
to the results attributed by CATTS to the chosen feedback category* 

APPLICATION OF CATTS 

The previous project has demonstrated the efficacy of CATTS instantaneous 
and delayed feedback in generating specific teacher behaviors in a preservice 
teaching environment. However, the system's potential for application in 
other training environments involving real time process analysis is virtually 
unlimited. Hence, the system is only a tool and with creative applications, 
cAn be used in many behavioral settings. 

The next logical extension of CATTS' methodology within training en- 
vironments is to access the CATTS system to inservice teanher training pro- 
grams within the schools themselves. This extension of the systim is in 
response to our concern for a cost-effective application of CATTS technology 
to the field of teacher training. CITH has undertaken and is presently 
analyzing the results of a number of remotely located inservice teacher 
training studies. The CATTS system is being used in some studies to collect 
data and provide feedback to teachers within school systems through the 
implementation of direct telephone and portable data recorder data collec- 
tion schemes. Data is collected in real*»time by a Touch Tone telephone 
input system and is delivered back to teachers immediately after teaching 
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via the sane telephone line to a teleprinter located within the school 
office • In other inservice studies teacher pupil interaction data is 
collected by observers with portable data recorders. The data is then 
mailed and processed at CITH and a printout feedback suHiarizing the class- 
rooo interaction is then returned to the teachers by nail* 

Various other applications using the CATTS system also suggest them- 
selves, such as extending current CATTS methodology into the field of 
regular education, and college and university teaching. For instance, 
feedback to professors concerning various student reactions or discussions 
related to classroom presented material woull serve to aid professors in 
their classroom planning and execution. This technology might thereby 
assist in attaining the goal of improvement of the quality of higher 
education. To move away from the field of teacher training, CATTS can be 
applied to one-on-one and small group situations such as counseling, 
clinical speech and hearing therapy, and clinical psychology sessions. In 
these situations feedback could provide additional process information to 
the session participants which would permit the monitoring or potential 
change of the direction or focus of the session. A further extension of 
this principle is to employ CATTS into biofeedback types of applications 
such as a sub:fect*s own monitoring of anxiety levels or other such measures 
of interest. These applications illustrate just a few of the additional 
areas in which CATTS methodology might be extended. 

In conclusion, the present paper has described a computer-assisted 
system which in real-time collects, analyzes, stores and feedbacks informa- 
tion based upon the application of various behavioral observation systems 
directed to the teacher training and teacher education field. In essence, 
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CATTS is a versatile and conprehensive delivery systen which can be applied 
in iiany ways within the teacher training field. CATTS can be of great 
assistance in the accooplishaent of training objectives for coapetency- 
or performance-based training prograas in teacher education. In our opinion, 
CATTS represents a quantum leap in teacher training in general. Moreover, 
with creative application, the system may be used in many ways to analyze 
snd feedback real-tine process variables within various behavior training 
fields. 
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MEAN RATE PERrORMANCE 



SELEaED 
FEEDBACK 
CATEGORY 



SECOND 
CHOICE 



Pi 

baseline 



TRHATMENT 



Printout only Tj ,1350 
Scope 5 printout T2 .1350 
XTiG Tj . .1350 



Printout only T^ .2675 
Scope 5 printout T-, .3492 
. XTi 5 T2 " .3083 



.2716 

.5017 

.3817 



.2175 
. 4417 
.3296 



INCREASE FROM 
BASELINE 



.1267 
. 3667 
.2467 



-.0500 
.0925 
.0213 



THIRD Printout only Tj .2908 .3542 .0634 

CHOICE Scope 5 printout Tj . 2303 .2267. -.0041 

XTi 5 T2 .2608 .2904 !o296 



Table 2. Mean rate performance on categories and treatment phases. 



17 



TEACHtNG STATION 



Z 

o 

< 

'o 



a 
o 
o 

I 



< 

> 
CE 
U3 

to 

CO 

■o 



observation 



classroom 



pupi Is 



c 



o o o 
o o o o 
bo 



coding 
box 



teacher 



T.V. monitoi/^ 
— 1 \/ 

4j 



' — ? 




I e lety pc- 



VI deo 
came ra 



y r ic] 



^ 

display 



\ 



■analysis-encoding station 



f j.g, 1, Schematic diagraa of present conceptual orrangcircnt of 
CATTS st&Lions. 



« 



18 




Figuri. 2. Meaji rate pcrfoixance by trpntmcnt across baseline and 
trcatndnt trials (TP interaction). 
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